作者：Jannis Vamvas Johannes Graën Rico Sennrich
我们介绍了SwissBERT，这是一个专门为处理瑞士相关文本而创建的屏蔽语言模型。SwissBERT是一个预先训练的模型，我们将其改编为用瑞士国家语言——德语、法语、意大利语和罗马语——撰写的新闻文章。我们在与瑞士相关的自然语言理解任务上评估了SwissBERT，发现它在这些任务上往往优于以前的模型，尤其是在处理当代新闻和/或罗马语Grischun时。由于SwissBERT使用语言适配器，因此在未来的工作中可能会扩展到瑞士德语方言。该模型和我们的开源代码在https://github.com/ZurichNLP/swissbert.
We present SwissBERT, a masked language model created specifically for processing Switzerland-related text. SwissBERT is a pre-trained model that we adapted to news articles written in the national languages of Switzerland — German, French, Italian, and Romansh. We evaluate SwissBERT on natural language understanding tasks related to Switzerland and find that it tends to outperform previous models on these tasks, especially when processing contemporary news and/or Romansh Grischun. Since SwissBERT uses language adapters, it may be extended to Swiss German dialects in future work. The model and our open-source code are publicly released at https://github.com/ZurichNLP/swissbert.
论文链接：http://arxiv.org/pdf/2303.13310v1
更多计算机论文：http://cspaper.cn/